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1.  INTRODUCTION 


The  proposed  project  centers  on  investigating  the  nature  of  upper  limb  prosthesis  use  in  everyday 
tasks  through  both  an  in-home  and  lab-based  study  on  upper-limb  amputees  and  age  and  gender- 
matched  normal  subjects.  For  the  in-home  study  we  will  use  an  unobtrusive  head-mounted 
camera  to  record  and  then  later  observe  prosthesis/hand  use  during  domestic  tasks.  In  the  lab 
study  we  will  use  a  motion  capture  studio  and  video  cameras  to  record  accurate  and  detailed 
upper  body  motion  during  a  series  of  standardized  tasks.  These  tasks  are  clinically  validated 
measures  of  hand  /  ann  function  functional  evaluation.  By  recording  participant  performance  and 
examining  prosthesis/hand  use,  we  expect  to  identify  shortcomings  in  current  terminal  devices 
and  implementations  that  will  infonn  improvements  to  existing  designs  and  inspire  new  classes 
of  devices  in  the  future. 

2.  KEYWORDS 

Upper  Limb  Prosthetics,  Amputee,  Assistive  Technology,  Motion  Capture 

3.  ACCOMPLISHMENTS 

This  reporting  period  covers  the  second  year  of  the  project.  This  portion  of  the  project  has 
involved  subject  recruitment  efforts  and  initial  data  capture  and  data  analysis,  using  custom 
software  tools. 

What  were  the  major  goals  of  the  project? 

The  major  goals  of  this  project  were  observing  the  upper  limb  manipulation  techniques  used  by 
numerous  upper  limb  prosthesis  wearers  and  ‘healthy’  individuals  (i.e.  those  with  intact  upper 
limbs)  when  achieving  a  variety  of  tasks  in  unstructured  (in  their  own  home)  and  structured  (in 
the  lab)  environments.  Comparing  data  from  these  two  demographics  over  the  different  tasks  and 
environment  we  aim  to  determine  differences  in  manipulation  techniques  between  prosthesis 
wearers  and  the  healthy  ‘baseline’.  In  particular  we  wish  to  identify  the  shortcomings  of 
particular  prosthetic  devices  or  setups  while  looking  for  methods  employed  by  prosthesis  users  to 
overcome  these  limitations.  These 

Originally  the  study  proposed  the  use  only  of  head-mounted  cameras  for  observation.  This  was 
extended  to  include  a  motion  capture  system  capable  of  accurately  recording  upper  body  motion 
to  provide  much  richer  movement  data.  The  motion  capture  setup  will  be  used  only  in  the 
laboratory  setup,  due  to  the  complexity  of  the  measurement  equipment  and  relatively  limited 
capture  volume. 
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What  was  accomplished  under  these  goals? 


In  the  first  year  we  prepared  measurement  equipment  and  the  necessary  protocols  to  enter 
participants  into  our  study.  In  particular  the  following  achievements  were  made: 

1 .  Experimental  protocols  were  finalized 

2.  The  protocol  was  approved  by  IRBs  for  all  institutions  and  the  DoD.  Necessary  human 
subjects  training  was  also  completed  for  relevant  members  of  the  study  team. 

3.  The  head-mounted  camera  setup  has  been  established  (a  modified  GoPro  Hero  3  with 
external  pocket  sized  battery  -  giving  6  hours  of  recording). 

4.  Software  to  aid  analysis  of  the  head-mounted  camera  data  was  prototyped 

5.  A  Vicon  optical  motion  capture  system  was  selected  (after  reviewing  several  options), 
purchased  and  installed  in  the  laboratory  space  of  Yale  University 

6.  Extensive  familiarization  with  the  Vicon  system  was  completed.  This  began  with  on-site 
training  from  a  Vicon  representative  but  since  then  has  led  to  the  following: 

a.  Optimized  camera  placement  (13  cameras  in  a  5x5m  space)  for  bi-manual  upper 
body  capture  when  standing  or  seated.  This  also  involved  installing  mounting  rails 
in  the  laboratory 

b.  Optimized  marker  placement  for  robustness  to  marker  occlusions  (when  motion 
capture  markers  are  hidden  from  view  in  particular  body  poses).  This  includes 
flexible,  wearable  marker  clusters  and  custom  software  methods  to  reconstruct 
occluded  markers. 

c.  Custom  data  processing  scripts  to  extend  the  functionality  of  Vicon  software  to 
export  skeletal  angles.  These  scripts  have  been  written  to  match  the  guidelines  of 
the  international  society  of  biomechanics  (ISB) 

7.  Collection  and  setup  of  materials  for  the  laboratory  space.  This  includes  a  variable  height 
desk  (to  simulate  a  kitchen  counter  or  work  desk)  and  various  household  items. 

The  setup  of  equipment  took  longer  than  initially  suggested  in  the  original  proposal.  This  was 
due  to  the  inclusion  of  the  motion  capture  system.  This  system  required  development  of  specific 
skills  and  significant  trial  and  error  regarding  camera  placement,  focusing  and  marker  sets. 


In  the  second  year  of  the  project  the  following  further  development  were  made: 

1 .  A  pilot  study  was  completed  of  the  at-home  study  with  a  healthy  non-amputee  volunteer. 
This  highlighted  problems  with  reliability  of  the  GoPro  remote  control  and  particular 
brands  of  memory  card. 

2.  Amputee  participant  recruitment  began,  with  various  advertisements  placed  in  amputee 
online  forums  and  social  media  sites. 

3.  Three  amputee  participants  were  recruited  within  Connecticut  and  New  York.  Two  were 
congenital  transradial  amputees  (one  male,  one  female,  both  body  powered  users)  and 
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one  was  a  non-congenital  shoulder  disarticulation  amputee  (who  uses  a  myoelectric 
prosthesis) 

4.  Custom  Video  Analysis  software  was  completed,  allowing  quick  and  robust  video 
tagging  by  use  of  a  midi  controller.  Exported  log  files  may  be  read  by  Matlab  or  Excel. 

5.  An  initial  ‘prosthesis  use  taxonomy’  was  created,  based  on  observation  of  the  video,  to 
allow  structured  recording  and  categorization  of  manipulation  events  observed  in  the 
recorded  videos. 

6.  The  recruited  amputees  all  took  part  in  the  at-home  study. 

a.  Several  hours  of  video  data  were  generated  for  each  participant 

b.  A  number  of  participant  videos  were  de-identified  via  blurring  of  portions  of  the 
video 

c.  Initial  video  tagging  was  completed  by  use  of  the  custom  software.  A  summer 
intern  was  hired  and  trained  for  this  task.  He  will  continue  to  work  with  us  in  his 
spare  time  for  the  remainder  of  the  project. 

d.  Initial  trends  were  observed  in  video  tagging  log  files,  via  Matlab  analysis. 

7.  Further  preparations  were  made  for  Motion  Capture  analysis,  including  a  full  pilot  study 
with  members  of  the  lab 

a.  An  additional  body  compensation  analysis  was  planned  and  piloted  on  members 
of  the  laboratory 


What  opportunities  for  training  and  professional  development  has  the  project  provided? 

The  project  provided  the  opportunity  for  familiarization  with  literature  on  prosthetics,  motion 
capture  and  functional  outcome  measures.  As  part  of  this  familiarization,  the  conference  MEC 
(Myoelectric  Controls  Symposium,  New  Brunswick  Canada)  was  attended  by  members  of  the 
project  during  the  first  year. 

Technical  training  was  completed  by  Dr  Adam  Spiers  on  the  Vicon  motion  capture  system. 
Training  was  also  completed  by  Dr.  Spiers  on  protocols  and  policies  regarding  human 
experiments.  Dr.  Spiers  has  subsequently  trained  two  grad  students  in  how  to  use  the  motion 
capture  system  and  written  a  guide  for  use  in  the  lab. 

As  a  result  of  the  at-home  studies,  Dr.  Spiers  has  become  familiar  with  running  studies  in  non¬ 
laboratory  scenarios. 

An  undergraduate  summer  intern  was  employed  for  video  tagging  and  video  de-identification 
(selective  scene  blurring).  This  provided  the  individual  with  an  understanding  of  the  functioning 
of  a  research  lab,  while  also  developing  their  video  editing  and  processing  skills. 


How  were  the  results  disseminated  to  communities  of  interest? 

Internal  dissemination  of  initial  findings  have  been  presented  to  our  lab.  We  have  not  discussed 
findings  outside  of  the  lab  yet. 
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What  do  you  plan  to  do  during  the  next  reporting  period  to  accomplish  the  goals? 

In  the  next  period  we  plan  the  following 

1.  To  recruit  more  amputee  participants  and  complete  at-home  studies  with  them. 

2.  To  recruit  gender,  age  and  height  matched  non-amputee  participants  and  complete  home 
studies  with  them 

3.  To  continue  tagging  of  the  at-home  video  data  using  the  custom  video  tagging  software 

4.  To  complete  in-lab  motion  capture  studies  with  all  recruited  participants 

5.  To  analyze  the  vicon  motion  capture  data  and  look  for  trends  in  the  data 

6.  To  disseminate  the  results  via  publications 

IMPACT 

What  was  the  impact  on  the  development  of  the  principal  discipline(s)  of  the  project? 

Describe  how  findings,  results,  techniques  that  were  developed  or  extended,  or  other  products 
from  the  project  made  an  impact  or  are  likely  to  make  an  impact  on  the  base  of  knowledge, 
theory,  and  research  in  the  principal  disciplinary  field(s)  of  the  project.  Summarize  using 
language  that  an  intelligent  lay  audience  can  understand  (Scientific  American  style). 


The  manipulation  taxonomy  developed  for  this  work  fills  a  gap  in  prosthetics  tenninology  that 
we  assume  will  be  used  by  other  researchers  in  the  future.  Such  manipulation  taxonomies  (e.g. 
the  Feix  taxonomy)  are  widely  used  in  healthy  human  and  robotic  hand  analysis,  yet  no  such  tool 
exists  for  prosthetics  use.  Though  Belter  et  al  created  a  ‘split  hook’  taxonomy,  this  was  not 
applicable  to  other  tenninal  devices,  such  as  multi-finger  hands.  We  have  designed  the  taxonomy 
to  be  generic  and  applicable  to  all  upper  limb  prosthetic  systems  and  levels  of  amputation. 


Similarly,  we  believe  the  body  compensation  measure  under  development  for  this  project  will 
also  provide  a  tool  that  may  be  useful  for  motion  analysis  in  research,  and  possibly  clinical 
setting.  Despite  body  compensation  being  a  known,  unwanted  factor  of  motion  impairment,  there 
is  no  universal  method  of  quantifying  the  level  of  compensation  for  particular  motions.  This  is 
addressed  by  our  kinematics  based  algorithm,  which  may  be  easily  added  to  a  motion  capture 
analysis. 

What  was  the  impact  on  other  disciplines? 

Nothing  to  report 

What  was  the  impact  on  technology  transfer? 

Nothing  to  report 

What  was  the  impact  on  society  beyond  science  and  technology? 
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Nothing  to  report 


5.  CHANGES/PROBLEMS: 

Changes  in  approach  and  reasons  for  change 

Addition  of  motion  capture  to  the  in-lab  portion  of  the  proposed  study. 

In  the  guidance  for  the  home  study,  we  specified  that  participants  are  not  permitted  to  spend 
more  than  30  minutes  on  sedentary  activities  such  as  watching  television  or  reading  a  book.  As 
this  may  have  been  abnormal  for  some  participants,  we  had  statements  from  some  volunteers  that 
they  ‘ran  out  of  things  to  do’.  A  guideline  document  of  suggested  activities  was  therefore 
provided,  including  such  tasks  as 

Actual  or  anticipated  problems  or  delays  and  actions  or  plans  to  resolve  them 

Year  1  -  Training,  setup  and  familiarization  of  with  the  motion  capture  system  added  delays  to 
the  project  compared  to  the  original  forecast.  However  we  believe  the  quality  and  impact  of  the 
resulting  data  will  be  much  higher  as  a  result  of  this  new  measurement  tool  and  the  time  taken  to 
leam  how  to  use  it. 

Y ear  2  -  Difficulties  in  participant  recruitment  delayed  the  start  of  the  at-home  study  and  has 
slowed  down  project  progress.  Typical  channels  of  subject  recruitment  (online  advertisements) 
did  not  generate  any  participants.  Instead  personal  connections  through  team  members  and/or 
their  colleagues  led  to  subject  recruitment  in  all  cases. 


Changes  that  had  a  significant  impact  on  expenditures 

Nothing  to  report 

Significant  changes  in  use  or  care  of  human  subjects,  vertebrate  animals,  biohazards, 
and/or  select  agents 

Nothing  to  report 


6.  PRODUCTS: 

Publications,  conference  papers,  and  presentations 

Nothing  to  report 

Website(s)  or  other  Internet  site(s) 

Nothing  to  report 
Technologies  or  techniques 

Motion  capture  marker  sets  and  processing  techniques  associated  have  been  developed.  These 
will  accompany  future  publications  as  appendices. 
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The  Midi  controller  based  video  tagging  software  developed  for  this  project  is  robust  and  easily 
scalable.  We  are  considering  open-sourcing  the  code  afterwards  for  use  by  other  researchers. 

The  prosthetics  use  taxonomy  is  a  manipulation  classification  technique  that  will  be  applicable  to 
general  analysis  of  upper  limb  prosthesis  use. 

The  body  compensation  algorithm  will  be  published  with  accompanying  data,  following 
completion  of  the  in-lab  study. 

Inventions,  patent  applications,  and/or  licenses 

Nothing  to  report 

Other  Products 

Nothing  to  report 

7.  PARTICIPANTS  &  OTHER  COLLABORATING  ORGANIZATIONS 
What  individuals  have  worked  on  the  project? 


Name: 


Project  Role: 


Researcher  Identifier  (e.g.  ORCID  ID): 


Nearest  person  month  worked: 


Aaron  Dollar 


PI 


Contribution  to  Project: 


Expert  on  human  hand  functional 
use  and  robot  /  prosthetic  hand 
development.  Contributed  to 
Protocol  development, 
measurement  equipment 
selection  and  setup. 


This  award. 


Funding  Support: 


Name: 


Project  Role: 


Researcher  Identifier  (e.g.  ORCID  ID): 


Nearest  person  month  worked: 


Linda  Resnik 


Co-PI 


Contribution  to  Project: 


Expert  on  upper  limb  prosthetics 
and  measures  of  upper  limb 
functionality  and  rehabilitation 
outcomes.  Contributed 
to  protocol  development. 


This  award 


Funding  Support: 


Name: 

Adam  Spiers 

Project  Role: 

Postdoctoral  Associate 

Researcher  Identifier  (e.g.  ORCID  ID): 

Nearest  person  month  worked: 

2 

Contribution  to  Project: 

Postdoc  researcher  responsible 
for  running  at-home  and  in-lab 
studies.  Contributed  to  protocol 
development,  IRB 
submission  (Yale  only), 
equipment  selection,  setup, 
customization  and 
familiarization. 

Funding  Support: 

This  award. 

Name: 

Kate  Barnabe 

Project  Role: 

Administrative  Lead 

Researcher  Identifier  (e.g.  ORCID  ID): 

Nearest  person  month  worked: 

1.2 

Contribution  to  Project: 

Protocol  development. 

IRB  submissions 

(all  institutions  and  DOD). 

Project  administration. 

Funding  Support: 

This  award 

Name: 

Paedyn  Gomes 

Project  Role: 

Undergraduate  Intern 

Researcher  Identifier  (e.g.  ORCID  ID): 

Nearest  person  month  worked: 

2 

Contribution  to  Project: 

Video  de-identification 

Video  Tagging 

Funding  Support: 

This  award 

Has  there  been  a  change  in  the  active  other  support  of  the  PD/PI(s)  or  senior/key  personnel 
since  the  last  reporting  period? 

Nothing  to  report 

What  other  organizations  were  involved  as  partners? 

Nothing  to  report 


8.  SPECIAL  REPORTING  REQUIREMENTS 
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A  Quad  Chart  accompanies  this  report 


9.  APPENDICIE S 

None 
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Actual  Expenditure:  $321,100 


